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Reference:  Government  Contract  No.  N00014-09-C-0050,  “Enhancing  Simulation- 

based  Training  Adversary  Tactics  via  Evolution  (ESTATE)” 

Charles  River  Analytics  Contract  No.  C08098 

Subject:  Contractor’s  Status  Report:  Quarterly  Status  Report  #2 

Reporting  Dates:  3/15/2009  -  06/15/2009 

Dear  Dr.  Hawkins, 

The  following  is  the  Contractor’s  Quarterly  Status  Report  for  the  subject  contract  for 
the  indicated  period.  During  this  reporting  period  work  has  concentrated  on  Task  1 : 
Identify  Training  Goals,  Task  4:  Develop  Trainee  Model  Processing,  and  Task  6:  Perform 
Simulation-based  Training  System  Integration 


1 .  Summary  of  Progress 

1.1  D&l  Program  Review 

A  program  review  was  held  2-3  JUN  2009  bringing  together  several  researchers 
funded  under  Dr.  Hawkins  in  the  areas  of  affordable  human  behavior  modeling  (AHBM), 
neuroplasticity,  and  human,  social,  cultural,  behavioral  (HSCB)  modeling.  The  goals  of 
this  program  review  were  to  bring  ONR  management  up  to  date  on  programmatic 
developments,  to  explore  new  opportunities,  to  provide  a  setting  to  encourage  possible 
future  collaborations  among  researchers,  and  gather  ammunition  technical 
accomplishments  and  transitions  to  showcase  the  program.  The  ESTATE  effort  was 
briefed  as  a  way  to  improve  training  through  simulation-based  training  technology  that 
provides  tailored  training  experiences.  The  high-level  overview  presented  was  a  way  to 
introduce  this  new  effort,  which  is  at  the  early  stages  compared  to  much  of  the  other 
research  presented.  While  there  was  no  immediate  overlap  in  this  research  with  the  other 
work  presented,  this  meeting  allowed  us  the  opportunity  to  collaborate  with  other 
research  institutions  in  the  near  future. 

1.2  Review  of  Training  Manuais  and  Guiding  Documents 

During  the  indicated  period,  we  were  able  to  obtain  training  manual  and  other  guiding 
documentation  from  PM  TRASYS  to  aid  in  establishing  training  requirements  and  gaps. 
The  first  document  reviewed  was  the  Irregular  Warfare  (IW)  Joint  Operating  Concept 
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Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


(JOCf  a  strategic  document  for  future  joint  force  commanders  (JFCs)  in  conducting 
protracted  irregular  warfare  on  a  global  or  regional  scale  against  hostile  states  and  armed 
groups.  The  IW  JOC  defines  irregular  warfare  as  “A  violent  struggle  among  state  and 
non-state  actors  for  legitimacy  and  influence  over  the  relevant  populations.  IW  favors 
indirect  and  asymmetric  approaches,  though  it  may  employ  the  full  range  of  military  and 
other  capabilities,  in  order  to  erode  an  adversary’s  power,  influence,  and  will.”  As  a 
result,  the  focus  becomes  the  legitimacy  of  a  political  authority  to  control  or  influence  a 
relevant  population,  a  focus  that  differs  from  conventional  warfare  of  defeating  an  enemy 
militarily.  When  referring  to  the  tactical  level  of  war,  the  IW  JOC  identifies  that  “the 
focus  is  probably  on  asymmetric  applications  of  tactics,  techniques,  and  procedures 
(TTP)  that  may  be  applied  differently  in  an  IW  operation  than  it  would  under  a 
conventional  operation.” 

The  IW  JOC  refers  to  the  2006  Quadrennial  Defense  Review  (QDR  2006)  that  helps 
define  the  concept  of  employing  “general  purpose  forces  (GPF)  continuously  to  interact 
with  aliens,  build  partner  capability,  conduct  long-duration  COIN  operations,  and  deter 
aggressors  through  forward  presence.”  In  regards  to  this,  the  IW  JOC  identifies  that  one 
of  the  major  risks  is  “operating  in  an  ambiguous  IW  environment  will  create  new  mental 
and  physiological  demands  on  personnel  conducting  long-term  assignments  in  foreign 
austere  settings.”  The  IW  JOC  recommends  a  mitigation  procedure  in  regards  to  this  risk. 

“Create  IW  models  and  simulations  using  live,  virtual,  and  constructive 
environments  that  provide  joint  force  leaders  and  members  the 
opportunity  to  practice  engagement  with  foreign  forces  and 
populations  in  daily  routine  activities  as  well  as  IW  operations. 
Exercise  communicating,  persuading,  and  negotiating  skills  using  the 
members’  language  proficiency  (or  lack  thereof),  cultural  knowledge,  and 
crosscultural  communications  skills.” 

Using  this  guiding  direction,  we  can  steer  the  ESTATE  work  towards  addressing 
these  goals,  focusing  on  interactions  instead  of  traditional  combat  operations.  The 
question  now  becomes  whether  the  original  proposed  research  under  ESTATE  is 
applicable  to  this  setting. 

In  addition  to  the  IW  JOC,  we  also  obtained  the  NAVMC  3500.44  Infantry  Training 
and  Readiness  (T&R)  Manual^,  the  established  Core  Capability  Mission  Essential  Tasks 
(MET)  for  training  the  Marine  Corps  infantry  battalion.  Of  interest  within  the  T&R 
Manual  is  the  use  of  simulation  codes,  which  indicate  whether  (A)  simulation  can  replace 
live  training,  (B)  simulation  supplements  training  before  field/live  fire  training,  (C) 
distance  learning  /  web-based  training  capability  exists,  or  (D)  no  simulation  is  available. 


'  Department  of  Defense,  Irregular  Warfare  (IW)  Joint  Operating  Concept  (JOC)  Version  1.0,  11 
September,  2007 

^  Department  of  the  Navy,  NA  VMC  3500.44  Infantry  Training  and  Readiness  Manual 
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The  Virtual  Battlefield  System  (VBS)  is  indieated  as  one  of  the  simulations  that  ean  be 
used  for  the  training  event.  We  are  currently  reviewing  this  document  to  see  which  METs 
are  (1)  appropriate  for  IW  training  and  (b)  can  be  supplemented  through  simulation.  We 
have  currently  identified  the  following: 

•  INF-MOBL-3 150:  React  to  an  unexploded  Improvised  Explosive  Device  (lED)  (A) 

•  INF-MOBL-3 152:  Defend  against  mines  and  booby  traps  during  motorized 
movement  (B) 

•  INF-MOUT-3801 :  Execute  upper-level  building  entry  (B) 

•  03 00-DEF - 1 002 :  Defend  a  position  (A) 

•  0300-MOUT- 1 00 1 :  Perform  individual  movement  in  an  urban  environment  (A) 

•  0300-MOUT-1002:  Perform  individual  actions  while  clearing  a  room  (A) 

•  03 00-PAT- 1005:  Perform  individual  movement  techniques  (A) 

•  0300-PAT- 1008:  Perform  individual  actions  in  a  patrol  (A) 

•  03 00-PAT- 1009:  Perform  immediate  actions  upon  contact  with  the  enemy  (A) 

•  0300-3568  PAT-1010:  Perform  individual  actions  from  a  vehicle  (A) 

•  0300-PAT-1014:  Perform  unaided  day/night  observation  techniques  (A) 

•  0311  -OFF- 1 00 1 :  Perform  individual  actions  in  a  fire  team  (A) 

Unfortunately,  the  T&R  Manuals  are  non-specific  in  regards  to  how  the  training 
should  be  performed  and  lack  key  indicators  of  measures  of  performance.  Furthermore, 
they  have  not  been  updated  to  accurately  reflect  the  needs  of  irregular  warfare.  We  plan 
to  review  with  PM  TRASYS  and  PM  TECOM  to  establish  how  these  guiding  documents 
can  best  be  used. 

1.3  Virtual  Battlespace  2  (VBS2)  and  OneSAF  v3.0  Obtained 

During  the  indicated  period,  we  were  also  able  to  secure  several  of  the  simulation 
environments  identified  during  the  prior  reporting  period  in-house.  Specifically,  Virtual 
Battlespace  2  (VBS2)  was  received  on  6  MAY  2009  as  part  of  the  Deployable  Virtual 
Training  Environment  (DVTE).  We  have  since  installed  VBS2  on  a  dedicated  machine. 
Unfortunately,  however,  the  distribution  of  VBS2  is  without  any  existing  training 
scenarios,  which  would  be  of  great  benefit.  Additionally,  the  existing  models  seem  sparse 
and  the  editor  functionality  has  not  been  fully  researched.  We  are  currently  looking  into 
the  scripting  capability  within  the  simulation  environment  for  providing  reactive 
behavior. 

During  the  indicated  period  we  also  received  OneSAF  v3.0  in-house  for  evaluation 
purposes.  Since  VBS2  is  of  higher  priority  as  a  target  for  integration,  we  have  postponed 
evaluation  of  OneSAF  until  VBS2  is  further  evaluated. 
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1 .4  Formal  Framework  for  Challenge  /  Response  Games 

During  this  reporting  period,  we  extended  our  work  in  the  conceptual  reformulation 
of  ESTATE  for  Challenge  /  Response  games.  Previously,  we  established  that  within  the 
challenge  /  response  framework,  a  challenge  problem  is  presented  to  the  trainee,  who 
then  attempts  to  response  to  the  challenge  in  the  appropriate  manner,  doing  so  either 
correctly  or  incorrectly  (a  graded  scale  of  performance  also  being  possible).  Based  on  this 
response,  a  new  challenge  can  be  generated  and  presented  next  that  promotes  learning. 
The  figure  below  represents  this  process  within  ESTATE. 


On-line  Execution 


We  took  this  conceptual  framework  and  then  began  to  apply  aspects  of  Item 
Response  Theory  (IRT)^  to  help  provide  a  computational  foundation.  In  IRT,  ability  is 
used  to  represent  and  measure  latent  traits  in  individuals  performing  a  function.  We 
represent  this  term  by  0.  While  0  can  range  from  positive  infinity  to  negative  infinity,  it  is 
typically  given  a  -3  to  3  range.  For  each  item  (or  challenge),  an  individual  has  a 
probability  of  getting  the  item  correct  or  incorrect.  This  probability  is  represented  by 
P(0).  Since  P(0)  is  a  function  of  0,  we  can  construct  an  item  characteristic  curve  (ICC) 
that  represents  the  probability  of  getting  an  item  correct  as  a  function  of  an  individual’s 
ability  level.  These  ICCs  are  normally  S-curves.  The  shape  of  these  S-curves  can  be 
defined  by  several  mathematical  models.  The  difficulty  of  an  item  is  a  location  index  that 
describes  where  the  item  functions  along  the  ability  scale.  For  our  purposes,  this  can  be 


^  Baker,  F.B.,  (2001),  Basics  of  Item  Response  Theory,  ERIC  Clearinghouse  on  Assessment  and 
Evaluation,  College  Park,  MD. 
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where  is  P(0)  =  50%.  The  discrimination  of  an  item  describes  how  well  the  item  can 
differentiate  between  examinees  having  abilities  below  the  item  location  and  those 
having  abilities  above  the  item  location  (essentially  the  steepness  of  the  ICC  in  the 
middle,  or  the  slope  of  the  line  where  P(0)  =  50%). 


Using  these  concepts,  we  can  think  of  the  ESTATE  conceptual  formulation  in  another 
way.  A  trainee  has  an  ability  level  at  any  given  time,  represented  by  0t.  Since  we  can 
never  know  the  true  ability  of  the  trainee,  we  can  only  estimate  it.  This  estimation  is 
assigned  0_hatt.  Via  simulation,  we  can  bring  the  trainee  ability  against  a  challenge  c  and 
come  out  with  a  result  r.  We  build  up  a  repository  of  these  interactions  as  a  history  of 
tuples  <c,  0_hatt.i,  r>.  During  diagnosis,  we  assess  the  current  estimated  skill  level  of  the 
trainee  based  on  the  history  of  traces  and  determine  0_hatt.  During  adaptation,  we  attempt 
to  find  the  optimal  challenge  c  that  will  promote  learning  to  serve  the  next  round,  c  can 
be  derived  from  finding  the  challenge  such  that  the  probability  of  getting  that  challenge 
correct  given  the  currently  estimated  ability  level  of  the  trainee  is  greater  than  or  equal  to 
the  probability  of  getting  the  probability  of  getting  that  challenge  correct  at  the  optimal 
abilitj^  level  minus  some  delta.  Formally,  Pc*(0_hatt)  +  AP  >=  Pc*(0  ).  We  can  assume  that 
Pc*(0  )  =  0-5,  since  at  the  target  skill  level,  with  the  optimal  challenge,  the  trainee  has  a 
50%  chance  of  responding  to  the  challenge  correctly.  Furthermore,  we  can  start  with  AP 
at  5%  or  10%  as  an  assumption  of  the  zone  of  proximal  development.  We  can  then  adapt 
AP  based  on  the  current  trend  in  answers  being  correct  or  incorrect  in  recent  history. 
Based  on  this,  60%  >=  Pc*(0_hatt)  >=  40%  with  a  AP  =  10%. 

Using  our  new  formulation,  we  can  adapt  the  above  diagram  to  represent  this  case. 
This  diagram  is  shown  below. 
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1 .5  Extracting  Knowledge  from  MoneyBee 

During  the  indicated  period,  we  have  begun  to  examine  MoneyBee,  a  money 
exchange  game  that  teaches  algebra,  logic,  and  estimation  previously  developed  under 
Brandeis  University’s  Bee  Web  program  (http://www.beeweb.org).  First,  we  began  by 
enumerating  all  the  possible  MoneyBee  problems.  Next,  we  ran  our  original  difficulty 
heuristic  for  each  of  these  problems.  We  are  currently  evaluating  these  results  to  see  if 
there  is  any  insight  to  be  gained  about  problem  characteristics  that  determine  difficulty. 
In  concert  with  those  efforts,  we  have  also  begun  to  collect  MoneyBee  data  from 
previously  played  games  to  an  effort  to  see  if  we  can  come  up  with  item  characteristic 
curves  for  each  problem.  Since  MoneyBee  isn’t  a  pass/fail  game,  we  have  cutoff  answers 
at  a  designated  period  of  time  to  see  if  the  answer  is  correct  or  incorrect.  The  results  are 
inherently  biased,  but  useful.  Next,  we  have  begun  to  setup  a  simulated  user  who  will  use 
the  ICCs  to  determine  whether  a  question  is  answered  correctly  or  incorrectly  to  answer  a 
question  of  a  given  difficulty.  We  hope  to  analyze  these  results  in  the  next  reporting 
period. 

2.  Scheduled  Items 

During  the  next  reporting  period,  we  plan  to  focus  on  the  following  tasks: 

•  Revisit  training  goals  with  PM  TRASYS  and  USMC  TECOM 

•  Reengage  with  VBS2  Program  Office  to  obtain  training  scenarios 

•  Examine  MoneyBee  data  to  determine  item  characteristic  curves  and  perform 
preliminary  experiments  with  simulated  user  data 
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•  Discuss  with  Program  Manager  change  of  focus  from  adversary  behavior  adaptation 
to  challenge  adaptation 

Sincerely, 


Brad  Rosenberg 
Principal  Investigator 
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